Optical navigation produces motion signals indicative of relative movement along the directions of coordinate axes, and is becoming increasingly prevalent. It is used, for instance, in optical computer mice and fingertip tracking devices to replace conventional mice and trackballs for the position control of screen pointers in windowed user interfaces for computer systems. It has many advantages, among which are the lack of moving parts that accumulate dirt and suffer the mechanical wear and tear of use. A more recent advantage of an optical mouse is that it does not need a mouse pad, since modern optical navigation techniques are generally capable of navigating upon arbitrary surfaces, so long as those surfaces are not optically featureless.
The modern optical navigation techniques mentioned in the preceding paragraph operate by tracking the relative displacement of images. A two dimensional view of a portion of the surface is focused upon an array of photo detectors, whose outputs are digitized and stored as a reference image in a corresponding array of memory. A brief time later a sample image is also digitized. If there has been no motion, then the sample image and the reference image are identical (or very nearly so). What is meant, of course, is that the stored arrays appear to match up. If, on the other hand, there has been some motion, then the sample image will appear to have shifted within its borders, and the digitized arrays will no longer match (that is, if their borders are also lined up). The matching process is termed xe2x80x9ccorrelationxe2x80x9d and may be performed in various ways, one of which is described in the incorporated Patents. Considered in isolation, correlation answers the narrow question xe2x80x9cAre these two images aligned?xe2x80x9d When the answer is xe2x80x9cNo,xe2x80x9d it could be because of intervening motion in any direction, and some additional mechanism is needed to find the direction and amount of displacement that will produce correlation results of xe2x80x9cYesxe2x80x9d or xe2x80x9cAlmost.xe2x80x9d What is done is to perform correlations between one of the stored images (say, the reference image) and a collection of shifted versions of the other (sample) image. The shifts are no shift, one over, one over and one up, one up, one over in the other direction, etc., for eight actual shifts and one xe2x80x9cnullxe2x80x9d shift. We would then expect that one of these nine correlations would be better than all the others, and its direction and amount of shift is taken as an indication of the intervening motion. Collectively, the nine correlation values describe a xe2x80x9ccorrelation surfacexe2x80x9d.
The optical navigation mechanism maintains velocity (speed and direction) information. When a new sample frame is to be correlated with the reference frame a predicted shift can be used as the starting point for the nine shifts. With this arrangement the correlation results contribute to an updated velocity and either ratify or modify the motion just predicted. Prediction is used to xe2x80x9cpre-shiftxe2x80x9d the reference frame, until such time as the new sample frame fails to overlap a significant portion of the shifted reference frame. At that time a new reference frame is taken. Prediction can be used to slow the sample rate down to conserve power (if the light source is pulsed, or can be turned off between samples) when the current velocity is low.
The basic technique described above works quite well when the surface produces images that appear as regions of light and dark defined by closed boundaries. The boundaries need not be abrupt; they can be gradual. And, if the shapes of the regions are sufficiently non-symmetrical or irregular, then any closure requirement for the boundaries can be relaxed. The shift and correlate mechanism may encounter difficulty, however, when there is a lot of symmetry and no closed boundaries. Say, for example, a mouse were navigating upon a wood grained desk. Under the right circumstances, the images presented to the correlator could appear to represent a series of parallel lines. The worst case scenario is that the spacing between the lines is regular and minimal, which obscures the effects of motion in a direction across the lines. Multi-bit resolution and interpolation can mitigate the problem, but these are not foolproof, and they are definitely not low cost. And, it is relatively difficult to detect motion in the direction along uniformly parallel lines. The result is an outfoxed mouse and a frustrated user. (To be fair, grainy surfaces are only sometimes troublesome. Much depends upon feature size, as well as orientation. Parallel features whose separation is less than the spacing between photo detectors are not resolved as separate entities. Difficult cases arise for parallel features that align with, say, every other (second) or every third photo detector. In general, the more the feature separation varies from those troublesome cases, the less difficulty arises. Bear in mind, then, that just because a surface is grainy does not mean that it can produce a difficult case.)
One solution to this occasional difficulty would be to increase the resolution with which images are represented. One might magnify the image more, increase pixel density, or digitize each pixel to a greater number of bits. The hope is that eventually what appears to be regular repeating (and thus indistinguishable) features become irregular (and thus distinguishable from one another). Each of these solutions has its own disadvantage, among which are increased cost and complexity, and a need for greater internal processing speed just to maintain tracking over the surface at a given rate of physical mouse motion.
Another technique that has been used to mitigate the grain/parallel line problem is (spatial) filtering the images before they are correlated. The idea is to modify the arrays to be correlated such that a regular pattern in the data is suppressed, at least in part. This allows the response of the correlation mechanism to be based more on irregular features in the image and less on regular recurring features that produce strong correlations not dependent upon motion. The idea is similar to filtering out a known strong signal to recover a weaker one. The filtering is performed by altering the values in the various positions of an array according to arithmetic combination with the values in symmetrically opposed positions disposed around the location of interest, and of necessity distorts or alters the image being tracked. This is acceptable, since we are not particularly interested in such fidelity, anyway. That is, we are never going to display to anyone the contents of the array as an image; we simply want to track it. The prior art contains different spatial filters usable in the manner we have just described, and once a suitable one has been identified by the designer it is simply incorporated as a permanent feature into the system.
Spatial filters have been used in optical mice to deal with the problem of parallel lines arising from, say, wood grain. They are generally effective in two orthogonal directions oriented relative to the spatial pattern with which the array of photo sensors is laid out, say parallel to the axes of the sensor array, or tilted at forty-five degree thereto. The incorporated ""544 Application describes two such filters, and a method of switching between them. The idea is that each filter has a limited range of operation and the mouse can be fooled by the appearance of long parallel features that are better dealt with by the other filter. The switching, or filter selection mechanism, operates in response to the detection of axial and diagonal ridges in the correlation surface (the AR, DR and RM metrics described in the ""544 Application).
Unfortunately, it is still possible for the system described in the ""544 Application to occasionally get confused by a grainy surface. In its defense, we note that the filter switching system was developed for use in a (very) low cost mouse optical navigation sensor, where part of the low cost was achieved by having the filter output be but a single bit (!) of resolution. This resulted in a considerable collateral reduction in complexity for the arithmetic circuitry needed for correlation. If multiple bit precision were retained it is quite likely that the performance of the multiple filter technique would improve correspondingly, possibly to the point where no further improvements would be felt necessary. As it is, however, multiple bit precision presently precludes the notion of very low cost (chip size goes up rapidly with all the additional arithmetic circuitry), leaving us to consider what might be done to further alleviate the effects of grain with only a modest increase in sensor cost. What to do?
A solution to the problem of optically navigating a mouse or the like upon grainy surfaces is to: First, equip the mouse with at least one additional optical navigation circuit whose field of view is different and whose axes are possibly oriented at an angle to those of the original optical navigation circuit and; Second, detect for each navigation circuit an indication of navigation impairment owing to grain, and; Third, navigate according to a suitable algorithm that chooses from among the various navigation circuits which one""s output to use. The multiple optical navigation circuits can each be separate self-contained mechanisms on separate dies, or preferably, they can be separate sections of a single integrated circuit. The various metrics used for filter selection can be further used by the navigation circuit selection algorithm. Each optical navigation circuit can have its own light source, or preferably, they can share a common one. Preferably, they also share any imaging optics. In any event, if the optical navigation sensors are oriented such that their respective axes are not parallel, then their indications of incremental (mouse) movement must be converted to a common framework before use. That conversion can be performed within the mouse as an adjunct to the navigation selection algorithm.